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1 ■  Derivation  of  the  Continuous 
Projective  SUMT  Method 


Without  Loss  of  generality,  a  nonlinear  programming  problem  can 
be  written: 


minimize  c  x 
n 

x€E 


subject  to  the  restrictions 


x  €  R  =  (x  |  g  <x)  2  0,  for  i  =  l,...,m}. 
i 


(If  in  the  original  formulation  the  objective  function  f(x)  is 
nonlinear,  adding  a  variable  z,  an  inequality  constraint  z  -  f(x)  2  0, 
and  minimizing  z  creates  an  equivalent  optimization  problem.) 

The  logarithmic  barrier  function  approach  for  solving  the  problem 
is  to  minimize 


* 

\ 

! 


I 
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R  =  (x  |  g  (x)  >  0,  for  i  = 
i 

for  a  sequence  of  values  r^  0. 

Under  certain  conditions  the  unconstrained  minimizers  of  the 
barrier  function  approach  solutions  of  the  original  problem. 

The  minimizing  trajectory,  denoted  by  x(r),  is  often  unique  and 
continuously  differentiable  as  a  function  of  r.  When  it  is  (see 
[Fiacco  and  McCormick  1968,  p.  77]), 

2  -1 

dx(r)/dr  =  9  P[x(r),r]  c/r.  (1.1) 

This  fact  has  been  very  helpful  in  the  development  of  extrapolation 

techniques  for  accelerating  convergence  to  the  solution. 

To  conform  to  the  usual  way  in  which  differential  equations  are 

written,  let  t  a  1/r,  and  define 

m  m 

2  2  T 

V(x)  =  -  l  7  g  (x) [1/g  (x)]  +.  I  7g  ( x) [ 1/g  (x)  ] Vg  (x)  . 

i  i  i  i  i 

i=l  i=l 

0 

Let  x  be  a  point  in  the  strict  interior  R  .  The  projective  SUMT 

method  is  to  apply  the  differential  equation  (1.1)  without  restricting 

x  to  be  an  unconstrained  minimizer  of  the  barrier  function  for  some 
0 

value  of  r.  Making  the  substitutions  above,  this  leads  to  the  . 
differential  equation 

-1 

dx/dt  a  -V(x)  c  (1.2) 

which  is  to  be  solved  from  the  initial  point  x(t  )  =  x  . 

0  0 
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Example 

Consider  the  problem: 


2 

minimize  x  +  y  subject  to  g(x,y)  =*  -x  +  y  A  0. 


Let  the  starting  point  be  (0,1),  where  g  =1.  The  differential  equa- 

0 

tion  to  be  solved  is 


/ dx/dt 
\dy/dt 


0\  2  -1 
o)  +!r> 


+ 


2  -2 

(-x  +y)  (-2x,l) 


(l+2x)(-x  +y) 

i  2  2 

L  (2x  +2x+2y)(-x  +y) 


(1/2) 


The  solution  is  (from  the  point  (0,1),  t  =  0) 


2  2 

x(t)  *  -t/[2(t+l)],  y(t)  =  [l/(t+l)  +  t  / [A(t+1)  ] 


The  trajectory  of  the  differential  equation  is  plotted  in  Figure  1.  It 
converges  to  the  solution  (-1/2,  1/4). 


2.  Convergence  to  a  Global  Solution  of 
the  Convex  Programming  Problem 

The  convergence  analysis  will  assume  the  following. 

0 

Assumption  1.  The  strict  interior  R  of  the  feasible  region  is  not 
empty. 

Assumption  2.  The  problem  functions  g  (x)  are  concave  and  twice 

i 

continuously  differentiable. 

Assumption  3.  The  set  of  points  such  that  x  €  R,  and 


is  compact. 


T 

f(x)  =  {inf  c  x  s.t.  x  6  R} 


Figure  1.  Solution  of  Example  Problem  by  Continuous  Projective 


Algorithm. 
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If  a  problem  does  not  satisfy  Assumption  3,  then  there  are  global 
minimizers  "at  infinity."  There  would  be  no  reason  to  expect  the 
algorithm  to  pick  out  a  finite  one  at  the  end  of  the  converging 
trajectory. 

Another  interesting  consequence  can  be  deduced  from  Assumption  3. 

Corollary  1.  Let  x  be  any  feasible  point.  Then 

0 

{x  |  f (x)  £  f(x  ) ,  x  €  R] 

0 


is  compact. 


Proof : 


The  proof  is  on  page  94  of  the  book  [Fiacco  and 


McCormick  1968] . 


The  importance  of  this  is  that  the  trajectory  x(t)  remains  in  a 
compact  set. 

Assumption  2  implies  that  the  matrix  V(x)  is  always  positive 
semi-definite.  In  order  for  the  algorithm  to  work,  positive 
definiteness  is  required. 

0 

Assumption  4.  For  every  x  €  R  ,  V(x)  is  positive  definite. 

This  assumption  is  not  limiting  and  is  usually  satisfied  in 
practice.  For  example,  if  the  linear  constraints  of  a  problem 
(including  some  nonnegativity  constraints  if  they  are  present)  have 
rows  which  span  En,  V(x)  is  positive  definite. 

The  next  assumption  is  traditional  in  nonlinear  programming. 


Assumption  5.  Let  x  be  any  point  in  R.  The  gradients  of  the 

0 

constraints  equal  to  zero  at  x  are  linearly  independent. 

0 

This  assumption  is  not  needed.  Use  of  it  facilitates  the 
exposition  of  the  proof  of  convergence. 


v  "/•.  'v\  V  v  v*  v 


6 


T-507 


The  next  well-known  theorem  on  bordered  matrices  is  used  in  the 
convergence  analysis. 

Lemma  1.  Let  A,  C  be  symmetric  matrices  and  assume  that  the 

-1  -1 

bordered  matrix  below  is  positive  definite.  Then  A  ,  C  exist  and 
are  positive  definite  and 


(0V> 


-l 

[BA  ,-I] 


where  D  is  positive  definite  and 


-1  T 

D  =  (C  -  BA  B  ) 

Proof:  omitted. 

The  following  well-known  approximations  are  also  needed. 
Lemma  2.  When  t  is  large, 

-1 

(It  +  B)  =  I(l/t)  +  o(l/t). 

Proof:  omitted. 


-1 

Lemma  3.  When  A  exists,  and  t  is  large. 


-1  -1  -1  -1 

(A  +  B(l/t) )  =  A  -  A  BA  (1/t)  +  o(l/t). 


The  following  notation  will  be  used  through  the  rest  of  the 


paper.  Let  G  »  diag(g  (x))  be  a  diagonal  matrix  consisting  of  the 

i 

T 

first  r  constraints.  Let  N  be  the  n  by  r  matrix  of  the  gradients  of 


the  first  r  constraints.  Let  S  be  an  n  by  (n-r)  basis  matrix 


generating  the  null  space  of  N.  (This  assumes  that  N  has  rank  r.) 

T  -2 

The  matrix  M  is  the  residual  when  N  G  N  is  removed  from  V.  Thus 


m  m 

2  2  T 

M  =  -  l  7  g  (x)(l/g  (x))  +  l  7g  (x)[l/g  (x) ]7g  (x) 
i  i  i  i  l 

i=l  iar+1 


Let  N#  denote  any  r  by  n  matrix  such  that  N(N#)  *  I. 


Hote  that  S  7g  <x)  =  0,  for  i  =  1 . r.  Note  also  that 

i 

T 

(N#)  7g  (x)  =  e  ,  for  i  =  1 . .  where  e  is  a  column  vector  with  a 

i  i  i 

one  in  the  ith  position  and  zeroes  elsewhere. 

In  terms  of  the  quantities  defined  above,  there  are  two  useful 

explicit  expressions  for  the  inverse  of  V.  The  first  is 

-  _2  T  T  “I  -1  /  T\ 

-1  T  -2  -1  G  +  (N#)  MN #,  (N#)  MS  /  (N #)  \ 

V  =  (N  G  N+M)  =  [N#,S]  {  } 

T  T  \  T  / 

S  MN#  ,  S  MS  J  \  S  / 


[N#,S] 


-2  T  -1 
J  +(N#)  MN#] 


'°) 


/  -2  T  -1\ 

/  [G  +(N#)  MN#]  \  -1  -2  T  -1 

+  (  I  [B  ] ( [G  +(N#)  MN#]  ,  -I) 


T  T  -2  T  -1  T 

where  B  =  [(S  MS  -  S  MN# [G  +(N#)  MN#]  CN#)  MS].  The  second  is 


[N#,S] 


0  0 


T  -1 
0  (S  MS) 


r  -i  i  i  /  t 

-1  T  T  -1  f  I  (N#) 

+  (E  ){-I,(N#>  MS(S  MS)  ]  >  I 

T  -1  T  ^  \  T 

(S  MS)  S  MN#  )  \  S 


-2  T  T  T  -1  T 

where  E  =  G  +  (N#)  MN#  -  (N#)  MS(S  MS)  S  MN#. 

These  were  obtained  using  the  bordered  matrix  inverse  formula 


from  Lemma  1. 


The  first  theorem  states  that  for  finite  values  of  t,  the  form  of 


the  differential  equation  is  such  that  no  constraint  value  can  be  equal 
to  zero. 

Theorem  1.  For  all  finite  t,  g  (t)  >  0,  for  i  = 

i 

Proof:  Assume  the  contrary.  There  is  a  finite  value  t  where 

at  least  one  of  the  constraints  equals  zero. 

It  is  easy  to  show  that  there  is  a  nonempty  set  I  of  the  subset 
of  the  integers  from  1  to  m  with  the  properties  that 


lim 

g  <t)/g  (t)  >  0, 

for 

all  i,j  in  I,  and 

t-*t 

i  j 

lim 

g  <t)/g  (t)  =  0, 

for 

all  i  e  I,  all  1  S  j  S  m 

t-»t 

i  j 

Intuitively  the  integers  in  I  are  associated  with  the  constraints 
tending  to  zero  at  t  at  the  same,  fastest,  rate. 

Without  loss  of  generality  assume  that  I  =  (l,...,r)}. 

-1 

In  order  to  prove  the  theorem,  an  approximation  to  V(t)  will 
be  made .  Bow 


2 

g  (t)  M(t) 
1 


m  m 

2  2 

=  -  l  V  g  <t)[g  (t)  /g  (t)]  +  l  Vg  (t) 
i  1  i  i 

i=l  i=r+l 


2 

g  (t) 

1 _ 

7 

g  (t) 


T 

Vg  (t)  . 
i 


This  tends  to  zero  as  g  (t)  tends  to  zero  because  of  (2.3). 


1 

the  approximation  from  Lemma  2, 


Using 


-2  T  -1  2  2 

(G  (t)  +  (N#(t))  M(t)N#(t))  =  G(t)  +  o ( g  (t)  ). 


Combining  this  with  (2.1)  and  the  approximation  from  Lemma  3, 


-1  T  •  -1 

B  (t)  =  [ IS  (t)M(t)S(t) ] 

T  -IT 

+  [S  (t)M(t)S(t) ]  (S  (t)M(t) (N//(t)  )  ] 

2  T  T  -1 

•  G(t)  I(N#(t)  )M(t)S(t)][S  (t)M(t)S(t) )  ] 

2 

+  o ( g  (t)  ). 

1 


The  full  approximation  is  then 


-1 

V  (t) 

T  -IT  2  T  TT-1T 

=  [N//-S(S  MS)  S  MM//]  [G  ][(N//)  -(N//)  MS(S  MS)  S  ] 

T  -1  T  2 

+  S(S  MS)  S  +  o(g  (t)  ). 

1 


From  this  it  follows,  for  i  =  l,...,r, 


2  2 

dg  /dt  =  -X  (g  )  +  o(g  (t)  ),  and 

i  i  i  1 


r 


2  2  T  T  -1  T  2 

df/dt  =  -  l  (X  )  g  -  c  S(S  MS)  S  c  +■  o(g  (t)  ) 
i  i  1 


i=l 

T  T  T  -1  T 

where  X  =  [(N//)  -(N#)  MS(S  MS)  S  ]c  is  the  vector  of  second  order 
estimates  of  the  K-K-T  multipliers. 

For  the  following  analysis,  the  zero  order  terms  are  ignored 
Since  every  term  in  (2.6)  is  nonpositive,  it  follows  that 


t  2 

f(t)  i  f(t  )  -  /  [X  (T)g  (t)]  dr. 
0  til 
0 


From  (2.5)  it  follows  that 


ln[g  (t)]  -  !n[g  (t  )] 
1  10 


t 

J  X  (  t  )  g  (  t  )  dx  . 
til 
0 


10 
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Since  by  assumption,  f  is  bounded  below  in  the  feasible  region,  the 
second  term  on  the  right  of  (2.7)  above  goes  to  a  finite  value. 
Therefore  the  right-hand  side  of  (2.8)  goes  to  a  finite  value.  If 

g  (t)  went  to  zero  as  t  ■»  t  the  left  hand  side  of  (2.7)  would 
1 

go  to  -®.  This  contradiction  proves  the  theorem. 

Let  x  denote  the  limit  of  x(t)  as  t  goes  to  infinity.  “Bar'’ 

notation  will  be  associated  with  other  quantities  evaluated  at  x. 

T 

Let  u^  denote  the  ith  component  of  (H#)  c.  These  will  be  shown 
later  to  converge  to  the  Ka rush- Kuhn- Tucker  multipliers. 

Lemma  2.  For  those  g  (t),  for  i  =  l,...,r, 

1 

lim  tg  (t)  >  0.  (2.9) 

i 
t 

Proof:  Using  the  previous  approximation, 

2  T  T  -1  T  2 

dg  /dt  =  -g  [u  -(c  S) (S  MS)  (S  MN#)e  )  +  o(g  (t)  ). 
i  i  i  i  i 

2 

The  quantity  multiplying  g^  above  has  a  finite  limit  as  t  ->  ®.  In 
order  for  the  above  limit  to  approach  zero,  l’Hopital's  rule  implies 
that 

2 

(dg  /dt)/g 
i  i 

goes  to  -«.  This  contradicts  the  fact  that  it  has  a  finite  limit. 
Corollary  2.  For  all  i  =  l,...,m,  (2.9)  holds. 

Proof.  The  first  r  constraints  go  to  zero  faster  than  any  of 

the  last  m-r  so  the  corollary  is  trivially  true. 


I 
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From  now  on  the  matrix  N  will  consist  of  the  gradients  of  all 

the  constraints  which  are  going  to  zero.  Assume  that  these  are 

g  , . . . ,g  .  The  matrix  M  has  the  form 
1  r 

m  m 

2  2  T 

M  =  -  l  V  g  (x)/g  (x)  +  l  Vg  (x)(l/g  (x)  )7g  (x)  . 

i  i  i  i  i 

i=l  i*r+l 

Because  of  Corollary  2,  the  matrix  M/t  has  a  finite  limit.  It 
may  tend  to  singularity,  but  this  doesn't  hurt  the  convergence  analysis. 
Because  of  Corollary  1,  f(t)  converges  monotonically  to  some 


lower  bound .  Thus 


T  -1 

lim  tc  V(t)  c  =  0. 


Two  results  of  this  are  (see  (2.1)  and  (2.2)) 


T  -2  T  -1  T 

lim  tc  N#[G  +  (N#)  MN#]  (N#)  c  =  0 


(2.10) 


T  T  -IT 
lim  tc  S(S  MS)  S  c  *  0. 


From  the  latter  it  follows  that 


T  T  -IT 
c  S(S  MS/t)  S  c  =  0. 


Because  M/t  has  a  finite  limit  (recall  (S  MS)  is  positive  definite) 
it  follows  that: 


lim  c  S(t)  =  0.  (2.11 

t 

L— a  3.  The  point  x  is  a  Lagrangian  stationary  point. 

Proof:  Because  of  (2.11),  i.e. ,  that  the  gradient  c  of  the 

linear  objective  function  is  orthogonal  to  the  basis  matrix  generating 
the  null  space  of  the  binding  constraint  gradients,  it  is  well  known 
that 


_T 

c  =  N  u 


for  some  vector  of  Lagrange  multipliers  u.  Because  of  the  linear 


independence  assumption  there  is  only  one  vector  and  it  is 


T 

u  =  (H#)  c. 


It  now  remains  to  show  that  the  components  of  this  vector  are 


nonnegative. 


Assume  the  g  's  are  ordered  from  "fastest"  to  "slowest”  in  the 
i 

rate  at  which  they  tend  to  zero.  Within  the  set  of  g  *s  tending  to 


zero,  there  are  possibly  two  groups.  The  first,  {g  . g  },  has  the 

1  P 

property  that 


2 

lim  t  g  (t)  =  0. 
t 


For  the  second  (i  =  p+1, . . . ,r) , 


2 

lim  t  g  (t)  >  0  (possibly  ®) . 

i 

t 


Let  G  represent  the  diagonal  matrix  associated  with  the  first  set, 


13 


T-507 


and  6  the  diagonal  matrix  associated  with  the  second  set.  Then 
2 


T  -2  T  -1  T 

t  c  (N#) [G  +(N#)  MN#]  (N#)  c  (which  goes  to  zero  from  (2.10)) 

2  T  -2  -12 

3  a  scalar  tending  to  zero  +  (u  )  [G  /t  +  D)  u  , 


2  T 

where  u  is  the  portion  of  (N#)  c  corresponding  to  the  constraints 

with  indices  p+l,...,r,  and  D  is  a  matrix  with  a  finite  limit.  The 
-2 

matrix  [G  /t  +  D]  is  positive  definite.  From  this  it  follows  that 
2 

_2 

u  =  0.  (2.13) 

That  is,  the  multipliers  associated  with  the  constraints  going  to  zero 

at  a  "slow"  rate  are  zero. 

The  exact  formula  for  dg  /dt  is  (see  (2.2)) 

i 

dg  /dt 
i 


/°  0  \ 

-I 

T 

=  c  (if# , S) 

(  1  + 

\  T  -1  / 

T  -IT 

\0  (S  MS)  / 

. (S  MS)  S  MN#_ 

-1 
[E  ] 


T  T  -1 
•  (-1,  (N#)  MS(S  MS)  ) 


T  n 
(N#)  Vg 

i 


T 

S  Vg 


l  J 


-2  T  T  T  -1  T 

where  E  *  G  +  (N#)  MM#  -  <M#)  MS(S  MS)  S  MM#. 

T  T 

Recall,  (N#)  Vg  =  e  ,  and  S  Vg  *  0.  Note  further  that  for 
i  i  i 

in<p, 


-1  2  T  2 

Be*  (0 . 0,tg  ,0,...,0)  +  o ( tg  (t)). 

i  i  p 


The  above  becomes 


T  T  -1  2  2 

dg  /dt  »  [u  +  c  S(S  MS)  MH#]g  +  o(tg  (t)). 
i  i  ip 

T 

Since  c  S  has  been  shown  to  tend  to  zero  (2.11),  it  follows  that  the 

dominant  term  is  -u  .  Because  g  is  going  to  zero,  this  ultimately 

i  i 

must  have  a  nonnegative  value.  Thus 

iO,  for  i  -  1, . . . ,p.  (2.14) 

Theorem  2.  The  point  x  is  a  global  solution  for  the  convex 
programming  problem. 

Proof:  Because  of  (2.12),  (2.13),  and  (2.14),  there  is  a  vector 

u  with  nonnegative  elements  such  that 

_T_ 

C  a  N  U 

_T 

where  N  is  the  matrix  of  the  binding  constraint  gradients.  Thus  the 
Karush-Kuhn-Tucker  sufficiency  conditions  are  satisfied  at  x. 

3 .  Linear  Equality  Constraints 
Consider  the  problem 

T 

min  c  x 

subject  to  g^(x)  i  0,  i  =  l,...,m 
Ax  a  b. 

The  modified  algorithm  to  handle  the  linear  equality  constraints  is 
given  below.  Let  n-K  be  the  rank  of  A. 


Let  B  be  a  basis  matrix  for  the  null  space  of  A.  The  modified 


algorithm  is  to  solve  the  differential  equation 


T  -IT 

dx( t) / dt  »  -  B[B  VB]  Be 


where  V  is,  as  before. 


m  m 

2  2  T 

-  I  V  g  <x)(l/g  (x))  +  £  9 g  ( x) ( 1/g  (x) )9g  (x). 

i  i  i  i  i 

i=l  i*l 


It  is  required  that  the  initial  point  x  satisfy  Ax  =  b,  and 

0  0 

g  (x  )  >  0,  for  i  = 
i  0 

The  proof  of  convergence  to  a  global  solution  of  the  problem 


above  when  the  functions  g  (x)  are  twice  continuously  differentiable 

i 

and  concave  is  a  straightforward  modification  of  the  proofs  just 


presented.  The  problem  is  to  be  regarded  as  constrained  in  a  subspace 


of  reduced  dimension. 


Specifically,  the  equivalent  problem  is 


T 

min  C  z  s.t.  G  (z)  2  0,  for  i  *  l,...,m, 
K  i 

z€E 


T  T 

where  C  =  c  B,  and  G  (z)  »  g  (x  B (z-z  )],  for  i  »  l,...,m. 

i  i  0  0 

The  assumptions  which  were  made  on  the  problem  in  x  space  are  now  to  be 


made  on  the  problem  in  z  space. 


The  changes  in  the  assumptions  are  straightforward.  The  only  one 


which  needs  explicit  clarification  is  the  assumption  that  the  "V" 


matrix  in  z  space  is  positive  definite.  This  matrix  has  the  form 
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m  m 

2  2  T 

-  I  V  G  (z)/(G  (z))  f  I  7G  (z) (1/G  (z))7G  (z) 
i  i  i  i  i 

i=l  i=l 


which  is  (using  the  chain  rule  of  differentiation  twice) 


m  a 

T  2  T  2  T 

-  I  B  V  g  (x)B/g  (X)  +  l  B  7g  (x)(l/g  (x) )Vg  (x)B. 
i  i  i  i  i 

i=l  i=l 


This  is,  under  the  concavity  assumption,  alt/ays  positive  semidef inite, 


but  the  additional  assumption  of  definiteness  is  required.  This,  as 


noted  before,  is  usually  satisfied  in  practice. 


Linear  Programming  and  Karmarkar' s  Algorithm 

One  form  of  a  linear  programming  problem  is: 

T 

min  c  x  s.t.  Ax  »  b,  x  *  0,  j  =  l,...,n. 

J 

In  order  to  apply  the  recent  projective  method  of  Karmarkar  [Karmarkar 
198A],  the  problem  must  have  the  form 
T  T 

min  c  x  s.t.  Ax  =  b,  e  x  =  n,  x  i  0,  for  j  =  l,...,n. 

He  further  assumes  that  the  minimum  value  equals  zero  and  that  the 
matrix  A  has  full  row  rank.  He  then  projects  x  into  another  space 
where  the  objective  function  is  nonlinear.  Attempting  to  minimize  the 
transformed  problem  yields  a  corresponding  iteration  in  the  x  place  of 


k 

where  D  =  diag(x  ),  a  is  a  step-size  scalar,  and  3  is  a  scalar  chosen 

j 

T 

to  ensure  that  ex  an.  (Here  e  is  a  vector  of  all  ones.) 
k-t-1 


_*.  l-. 
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T 

If  the  requirement  e  x  =  n  were  ignored,  KarmarVcar's  direction 


would  be  proportional  to 


T  2  T  -1 

s  -  -Dll- (DA  ) [AD  A  ]  (AD) ]Dc. 
1 


The  direction  given  by  the  Projected  SUMT  algorithm  modified  for 


linear  equality  constraints  is 


T  -2  -IT 

s  =  -D[B  D  B]  Be, 
2 


where  B  is  a  basis  for  the  null  space  of  A. 


It  will  now  be  shown  that  these  two  vectors  are  the  ~ame.  (The 


similarity  between  directions  generated  by  Karmarkar's  algorithm  and 


those  given  by  the  projected  form  of  the  logarithmic  barrier  function 


method  was  first  pointed  out  in  (Gill,  et  al.  1985]).  Now 


2  2  T  2  T  -1  2 

As  =  -AD  c  MAD  A  )(AD  A  )  AD  c  =  0,  and 
1 

T  -2  -IT 

As  =  -AS [S  D  A]  S  c  =  0. 

2 


Furthermore 


T  -2  T  T  T  2  T  -1  -2  T 

BD  s  a-Bc+BA  (AD  A  )  AD  c  =  -B  c,  and 
1 

T  -2  T  -2  T  -2  -1  T  T 

B  D  s  =  -(B  D  B)(B  D  B)  B  c  =  -B  c. 

2 


It  is  necessary  to  show  that  each  row  of  A  is  linearly 
T  -2 

independent  of  B  D  .  If  not,  there  is  some  row  (w.l.o.g.  the  first) 


for  which  a  vector  d  exists  such  that 


T  T  T  -2 
e  A  =  d  B  D 
1 


Multiplying  both  sides  on  the  right  by  Bd  yields 


T  T  T  -2 

0  =  e  ABd  =  d  B  D  Bd. 
1 


T  -2 

Since  B  D  B  is  positive  definite  this  implies  d  =  0,  a  contradiction. 
Now 


.  (s  -s  )  =  0, 
T  -2  /  1  2 

kB  D 


and  the  matrix  has  rank  n.  Thus  the  vectors  are  equal. 


4 .  Discussion 

It  is  not  being  argued  that  the  Projective  SUMT  method  is  exactly 

equivalent  to  the  algorithm  proposed  by  Karmarkar.  But  the 

similarities  are  striking,  and  the  amount  of  work  required  to  compute 

the  direction  vectors  seems  equivalent. 

Several  points  are  worth  noting.  First,  the  convergence  of  the 

Projective  SUMT  method  depends  on  several  reasonable  assumptions  which 

are  usually  satisfied  in  practice.  It  does  not  depend  on  the  value  of 

the  objective  function  being  equal  to  zero  at  the  solution,  or  on  the 

existence  of  some  potential  function  which  is  being  minimized  at  each 

iteration.  The  reason  that  the  algorithm  never  encounters  the  boundary 

of  the  feasible  region  for  finite  t  is  that  the  form  of  the  V  matrix, 

2 

with  the  (g^)  's  occurring  in  the  denominator,  forces  a  slower  rate  in 
the  change  of  these  constraint  values  as  the  boundary  is  approached. 

Work  is  in  progress  to  describe  a  discrete  version  of  the 
Projective  SUMT  method.  The  most  likely  form  it  will  take  is  to 
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generate  at  x.  the  direction  of  search 


-1 

s  =  -V(x  )  c. 

k  k 


-1 

The  next  point  isx  *  x  -  V(x  )  ct,  where  t  is  the  minimizer 

k+1  k  k  k  k 

of  the  function 


m 

-*n[f(x  )-f[x  < t> ] ]  -  l  Inlg  (x  (t))J, 
k  k  i  k 

i=l 


0 

where  x  (t)  =  x  +  s  t  is  restricted  to  remain  in  R  .  (Here,  of 
k  k  k 

T 

course,  f (x)  =  c  x. ) 


The  step-size  function  is  a  variant  of  the  method  of  centers 

T 

function  proposed  in  [Huard  1967].  Because  s  7f(x  )  <  0,  the 

k  k 

derivative  of  the  function  above  gives  an  infinite  impulse  to  the  term 


associated  with  the  objective  function  and  f  decreases  for  t  small.  As 


the  point  x  (t)  approaches  a  constraint  boundary,  the  function 


increases.  Because  the  step-size  function  is  convex  (and  usually 


strictly  convex)  there  is  a  unique  solution  to  the  step-size  problem. 


It  is  not  obvious  to  the  author  that  any  discrete  version  of  this 
algorithm  will  be  superior  to  the  ordinary  logarithmic  barrier  function 
method.  The  main  reason  for  the  abandonment  of  the  latter  was  the 


ill-conditioning  of  the  Hessian  matrix  of  second  derivatives  of  the 

barrier  function.  The  same  problem  arises  in  inverting  the  V-matrix  of 

the  Projective  SUKT  algorithm.  To  date,  however,  no  one  has  tried  the 

approximate  inverse  as  given  by  (2.4).  The  usual  approach  has  been  to 

form  the  matrix  and  then  invert  it.  By  separating  out  the  g  *  s,  which 

i 
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T  -1 

are  small,  and  computing  the  associated  H#,  S,  and  (S  MS)  ,  the 
numerical  problems  are  no  more  difficult  than  those  ordinarily 
encountered  in  nonlinear  programming.  The  fact  that  the  Hessian 
matrices  are  usually  dyadic  in  structure  (see  [McCormick  1983,  Chapter 
3])  can  be  used  to  advantage  also. 

It  is  not  obvious  that  any  discrete  version  of  this  algorithm  is 
superior  to  a  second  order  “direct”  method  for  nonlinear  programming 
problems  (see  [McCormick  1983,  Chapter  15]  for  a  development  of  such  a 
method).  Indications  from  people  who  have  experimented  with 
Karmarkar’s  method  are  that  in  the  initial  phases,  his  method  can 
bypass  the  combinatorial  problems  arising  in  movement  from  vertex  to 
vertex.  That  is,  in  his  method,  as  well  as  in  that  just  described,  all 
the  constraints  influence  motion.  In  direct  methods,  only  constraints 
binding,  or  nearly  binding,  enter  into  the  computations.  If  this  is 
the  reason  for  some  of  the  success,  it  argues  for  better  “active  set 
strategies"  in  traditional  linear  and  nonlinear  programming  algorithms. 

To  answer  the  question  about  which  algorithms  will  in  the  long 
run  be  “better"  will  require  much  computational  testing,  and  much  work 
on  the  matrix  methods  needed  to  implement  them. 
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